The goal of this research is to develop novel machine learning techniques to integrate physician expertise and machine learned, logical rules in a graphical model that will accurately estimate breast cancer risk after breast biopsy. Our multidisciplinary team has a track record (including NIH funding and publications in the medical and computer science literature) illustrating an innovative research program that merges cutting edge machine learning algorithms including inductive logic programming and statistical relational learning to train graphical models to predict breast cancer risk. However, in contrast to prior work, we are testing a completely new methodology which we call Advice-Based-Learning (ABLe). By developing ABLe, our team aims to establish an innovative, collaborative cycle between machine-learning and physician expertise. We propose to test the hypothesis that this cycle will increase accuracy beyond what either the machine or human can accomplish alone. Specifically, we hypothesize first that a conventionally-trained graphical model trained with conventional machine learning first algorithms can accurately predict the risk of breast cancer after core biopsy and perform better than current clinical practice;a critical aim that is favorably foreshadowed by our new preliminary data but is labor intensive because we must perfect our unique clinical data that accurately represents clinical experience. Second, a graphical model trained using ABLe can incorporate multi-relational data with physician expertise and significantly improve the predictive accuracy over conventionally trained graphical models and current clinical practice. Third, our best graphical model trained with ABLe will accurately estimate the probability of malignancy after breast biopsy on new clinical cases better than physicians alone resulting in a tool that has the potential to improve care. Our clinical application is as compelling as our algorithmic work. Image-guided core needle biopsy of the breast is a common procedure that is imperfect, has high-stakes, and is particularly amenable to improvement with automated decision support. Breast core biopsy, the standard of care for breast cancer diagnosis, can be "non-definitive" in 5-15% of women undergoing this procedure. This means that between 35,000-105,000 women will require additional biopsies or radiologic follow-up to cement a diagnosis and risk the possibility of missed breast cancers, delays in diagnosis, and unnecessary surgeries. This important problem is emblematic of a plethora of clinical situations where rigorous and accurate risk estimation of rare events provides the opportunity for automated decisions support tools to personalize and strategically target health care interventions to improve decision-making for health-care providers and patients. This award will enable us not only to produce graphical models that provide improved decision support in the breast cancer clinic, but also, and more significantly, to develop a methodology that integrates heterogeneous predictive data and physician knowledge within a graphical model, thereby developing and validating a new algorithmic paradigm for creating accurate, comprehensible, adaptable decision support tools well-suited for clinical translation.